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I. INTRODUCTION 


The goal of this thesis is to present a method for determining 
whether office automation (OA) increases productivity in Naval 
Laboratories and whether OA is cost effective for those laboratories. 
To this end the Naval Laboratories Technical Office Automation and 
Communications System (NALTOACS) Pilot Assessment and the office 
automation pilot system at Naval Weapons Center, China Lake, 
California have been analyzed. An overview of the major issues of this 
thesis 1s contained in this chapter. 

This study addresses three major issues: productivity, the costs 


and benefits (cost/benefit) of office automation, and the methodology 


used to determine productivity and its cost/benefit. The first issue, 
productivity, 1S a measure of efficiency and ne ee ee (Greenwood, 
1984). Efficiency involves camparing inputs to outputs: how much or 
how little input is required for acceptable output or conversely how 
much output is acceptable given a fixed level of inputs. Effectiveness 
involves attainment of goals: for example, how a program or system 
affected profits if profits were the organizational goal. The second 
issue, cost/benefit, is addressed by comparing the overall benefit 
attained from the system with the overall cost. This is accamplished 
after a measure of office productivity has been determined. The third 


issue, the methodology used to determine productivity and cost/ 


benefit, is addressed through analysis of the methodology used by 





communications, computerized branch exchanges, and data base 


Management systems (Levine, 1985). 


B. POTENTIAL BENEFITS OF OFFICE AUTOMATION 

Computer technology was initially sought by industry as a means of 
cutting secretarial and clerical costs. Office employment is estimated 
to be 30% of the total labor force (Panko, 1985). Greenwood (1984) 
estimated that 35% of the office labor force is involved in 
secretarial or clerical work. Given these estimates, secretarial/ 
clerical work is estimated to canprise approximately 10% of the total 
labor effort in the workplace. Office automation has clearly proven 
that 1t can increase secretarial efficiency through the use of word 
processing equipment. Law and Pereira (1976) have shown through 
experimentation that the average secretary can only type ten words per 
minute when errors and disturbances were factored in. In their 
experiments, word processing equipment use was judged responsible for 
secretaries increasing their output to thirty words per minute: a 300% 
increase in output. Additionally, Shiff (1974) proposed that 
implementation of canputerized office equipment could eventually 
eliminate typing, transcription, and filing from office work: thus 
eventually eliminating the secretary, too. 

Apart from the reduction in secretarial help, Gottheimer (1979) 
suggests that automation of the office would significantly reduce the 
need for middle managers. This can be done by making executives more 
efficient. Macfarlane (1983) sights several aspects of OA systems that 
are important to management productivity. Information retrieval can 


save executives time py allowing them to access personal and corporate 


information from their desk. Analytical tools such as spreadsheets can 
be used to analyze information once it has been retrieved. Special 
applications programs such as graphics allow presentation of 
information in a manner that makes reports more persuasive. Electronic 
mail allows information exchange in a faster and more informal fashion 
than normal mail which leads to a greater exchange of ideas and 
information. 

In addition to increasing efficiency, OA can have other tangible 
and intangible benefits. Williams and Jones (1984) list the following 
as tangible benefits: 

- equipment charges eliminated 
- temporary help not needed 
- overtime reductions 
- work force reductions 
They also list the following as intangible benefits: 
- improved canmunications 
- enhanced document appearance 
- less time to prepare proposals and reports 

Additionally, the Honeywell Technalysis Office Management Systems 
Division conducted a national survey in 1985 to examine the attitudes 
of office workers who have access to OA equipment. That survey 
revealed that 90% of those surveyed reported that due to OA they had 
more time to devote to creative activities. Those surveyed also 
reported that OA played a major role in helping them make better and 
more informed decisions. About half of those surveyed stated that OA 
equipment is responsible for increases in speed, efficiency, and 
productivity. 

OA has been identified as a means for organizations to increase 
their productivity while decreasing their costs (Forest, 1979; 


Gottheimer, 1979; Williams and Jones, 1984). 
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C. IS OFFICE AUTOMATION REALLY BENEFICIAL? 
There 1s a fear throughout industry that the potential benefits 
extolled by OA champions will not be realized. The installation of new 
technology per se is not the way to make offices more efficient and 
productive, according to Charles Callahan (EDP, 1985). Callahan 
contends that office autamation cannot be justified by the vague claim 
that OA will "increase productivity" (p. 4); "management wants bottom 
line payoffs" (p. 4). Ralph Kleim (1985) wrote that using OA systems 
does not necessarily mean a bona fide increase in productivity. 
Kathleen Foley Curley (1984) summerizes the skepticism about 
office autamation. 
A real fear, particularly among line managers, is that new office 
automation technologies "may make us the best informed, unprofitable 
company in our market." How much infommation is too much? what are 
the benefits to be gained and how can they be quantified? There is 
also a question of whether an investment in office autamation 
technology 1S appropriate in every business or in every industry. 
ie. 37) 

The notion that OA may not be appropriate for every business or in 

every industry leads to the following question: how can a business or 

industry determine whether OA is appropriate for their needs? 


Specifically for this thesis, how can Naval Laboratories determine 


whether possible OA benefits will be realized? 


D. OFFICE AUTOMATION AND NAVAL LABORATORIES 

The Navy currently has eight laboratories that perform a wide 
variety of research functions. The goal of these laboratories is the 
development of new or improved equipment and/or techniques to support 
of the missions of the Department of Defense. 

As early as 1979 then Secretary of the Navy (SecNav), W. Graham 
Claytor, Jr., stated in a memorandum to the Chief of Naval Operations 
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that administrative modernization in the Department of the Navy was 
required to cope with "the volume of correspondence and administrative 
material" (SecNav, July 1979). In 1982 the Naval Laboratories 
Technical Office Autamation and Communications System (NALTOACS) 
program was established to coordinate the introduction of office 
systems technology into Naval laboratories as a means of enhancing 
productivity (NALTOACS, 1983). Previous to the establishment of 
NALTOACS, three of the eight Naval Laboratories had introduced OA 
technology. They are: 

- David W. Taylor Naval Ship Research and Development Center 

(NSRDC), Bethesda, Maryland 


- Naval Surface Weapons Center (NSW) 
Dahlgren, Virginia 


- Naval Weapons Center (NW) 
China Lake, California 

David W. Taylor NSRDC was assigned as the lead laboratory and was 
given the responsibility for NALTOACS. The Deputy Under Secretary of 
the Navy for Financial Management reaffirmed a requirement from the 
Commander, Naval Data Automation Cammand that an assessment of the OA 
pilot projects already in place should be performed prior to the 
initiation of any further OA implementation in Navy Laboratories. The 
pilot assessment done by David W. Taylor NSRDC was campleted without 
gathering any quantitative evidence (1.e., reduction in personnel 
costs or travel costs) relating to productivity in an autamated 
office. The study drew inferences of productivity changes by gathering 
subjective information (1.e., user opinions and estimates of how much 


time per day the OA system saved them) fram laboratory OA users. The 
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Naval Laboratory personnel were surveyed and asked to estimate their 
perceptions of productivity gains made possible through the 
installation of the OA system. The results of the pilot assessment 
done by David W. Taylor NSRDC were published in the "Pilot Assessment 


Final Report" dated April 29, 1983. 


&. OFFICE AUTOMATION AT NAVAL WEAPONS CENTER (NWC), CHINA LAKE 

NWC, China Lake is the principle Navy research, development, test 
and evaluation center for the air warfare, electronic warfare, missile 
systems, defense suppression, and parachute systems. The need for 
office automation technology at NWC China Lake was documented first in 
Proposal for Establishment of an Autamated Technical Information 


Processing System (ATIPS) (NWC, July 1980). This proposal sighted 


"heavily increasing paper workloads along with reductions in the 
manpower currently handling those workloads" as a major problem at 
NWC. Additionally, delays in document production and problems with 
copying and storage of "phenamenal amounts of paper" contribute to the 
waste of "hundreds of thousands of dollars each year" (NW, July 
1980). 

To handle this problem two courses of action were recanmended: 
"(1) make each person at NWC capable of doing more work with less 
effort in shorter periods of time and (2) reduce dependence upon paper 
as a medium of canmunication" (NW, July 1980, p. 2). 

The ATIPS Concept Development Paper (NWC, March 1982) delineated 
the functions necessary of the ATIPS system to resolve the problems 


brought forth in the Proposal for Establishment paper (NW, July 


1980). To be effective ATIPS would need to provide the following: 


Is 


Text Processing: word processing, text entry, document 
editing and revision, special forms/reports production. 


Records Management: filing, indexing, and retrieval of 
information on request. 


Information Processing: analytical models for analysis/ 
camputations/projections, file manipulation, text/data 
merging, business graphics. 


Management Information: allow access to internal and 
external data sources. 


Communications: provide inter-office connection for 
information exchance. 


Personal Services: office calendar, daily schedule, 
office directory, personal records handling capability. 


Given equipment that could fulfill the functional requirements 


listed above the ATIPS system was expected to provide the following 


benefits: 
l. Reduce administrative workload for technical and administrative 
personnel. 
2. Provide ready access to large quantities of information. 
3. Reduce the space required for storage of information. 
4. Reduce administrative costs. 
5. Reduce the number and size of meetings. 
6. Reduce the overall labor cost for each productive unit of work. 
7. Make NWC a more cost effective Research, Development, Test and 


Evaluation (RDT&E) center. 


The foundation of the ATIPS system is the Xerox 8000 STAR system. 


This system was in its developmental stage when it was delivered to 


NWC. The software installation was provided with the Xerox system. The 


telecommunications equipment was provided by Xerox, too; the 


teleconmmunications equipment was also still in the developmental 


Stage. Thus, NWC acted as a test site for the Xerox Ethermet 


14 


(NALTOACS, April 1983). The ATIPS system was initially operational in 
February 1982. The system was scheduled to grow to approximately 1500 
terminals to be used by 2500 people; the total cost of the system was 
budgeted at $11.875 million (NWC, August 1981). However, ATIPS funding 


was not continued beyond the installation of the pilot system. 


F. SUMMARY 

Evidence indicates that there are potential benefits that can be 
realized by organizations through the implementation of office 
automation systems. The primary benefit claimed by OA is that 
productivity will increase while cost, in the long term, will be 
reduced. However, a survey of current literature indicates that there 
is skepticism over the actual realization of OA benefits. The 
Department of the Navy has established an organization (NALTOACS) to 
supervise the implementation of OA systems in Navy Laboratories. 
However, prior to the implementation of OA systems in Naval 
Laboratories evidence should be presented that specifically supports 
the claims that OA increases productivity in the Naval Laboratory. The 
Pilot Assessment done by NALTOACS included information on the 
productivity changes experienced by the three pilot programs surveyed. 
However, this information was gathered in a qualitative manner. Naval 
laboratory personnel were asked to eStimate their perceptions of 
productivity gains made possible through the implementation of OA 


systems. 
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II. NALTOACS PILOT ASSESSMENT REVIEW 


Chapter I gives an overview of office automation, its potential 
benefits, the current status of OA in Naval Laboratories, and the 
status of the office autanation system at NWC China Lake. A brief 
overview of the Pilot Assessment of the three Naval Laboratory OA 
pilot projects, done by NALTOACS, is also presented. 

This chapter analyzes the NALTOACS Pilot Assessment in detail. 
This analysis is not concerned with the results of Pilot Assessment 
but rather with the method by which NALTOACS performed their 
assessment. To perform an analysis of the NALTOACS evaluation method 
it is necessary to begin by addressing evaluations in a very general 
sense: what they are, what their purpose is, and why they are useful. 
Next, the two main types of evaluations, quantitative and qualitative, 
are explained including their strengths and weaknesses. The issue of 
evaluation objectivity vs. subjectivity is also discussed. 
Additionally, this chapter explains why the NALTOACS Pilot Assessment 
is an inadequate evaluation of the impact of office automation 


technology in Naval Laboratories. 


A. EVALUATION DEFINITION 

What is an evaluation? Deming defines evaluation as a 
pronouncement concerning the effectiveness of some treatment or plan 
that has been put into effect (Guttentag and Struening, 1975). It isa 
“process used to analyze the relationship between actual and desired 


effects" (Euske, 1984, p. 53). Evaluation is the measurement of 
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performance and the making of camparisons based on those measurements 
(Wholey, 1979). "Evaluation is the measurement of desirable and 
undesirable consequences of an action intended to forward some goal 
that the actor values" (Riecken, 1953, p. 1). All of these definitions 
have measurement and comparison as cammon ground. An evaluation may be 
searching for the effector of performance or an evaluation may be 
searching for the effect that an action has on performance; in both 
Situations there is commonality in necessity for measurement and 
comparison. For the purpose of this thesis Riecken’s definition of 
evaluation will be used. 

Riecken ‘s definition of evaluation is very broad and general. This 
thesis is concerned with "action" defined as the installation of an 
office autamation system. In the following sections of this chapter 


this definition is furtner refined. 


B. PURPOSE OF EVALUATION 
Anderson and Ball (1978) list six major uses and neeas for 
evaluation: 
1. To contribute to decisions about program installation. 


2. To contribute to decisions about program continuation, 
expansion, or certification. 


3. To contribute to decisions about program modification. 
4. To obtain evidence to rally support for a program. 
5. To obtain evidence to rally opposition to a program. 


6. To contribute to the understanding of basic psychological, 
social, and other processes. (pp. 3-4) 


This thesis is most concerned with purposes 1, 3, and 4. These three 


purposes are all basically the same in substance but with shades of 
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difference in timing and user/evaluator orientation or bias. The issue 
of timing involves whether a program is 1n place or not; purpose one 
is oriented toward a new program that is not yet in place while 
purpose three and four are oriented toward a program that is already 
in place. The issue of user/evaluator orientation or bias involves the 
objectivity of the user/evaluator; purpose one is concerned with 
gathering evidence in an objective manner while purpose three and four 
are looking for evidence in a positive or negative sense. Funda- 
mentally, all three of these purposes can be condensed into a single 
concept: the purpose of an evaluation is to obtain evidence about the 
effectiveness of a program. 

The purpose of evaluation that I propose above is very broad and 
general. From this point forward this thesis will narrow the term 
"program" used in the explanation of evaluation purpose to "the 
installation of office automation technology." Therefore, for this 
thesis the purpose of an evaluation is to obtain evidence about the 


effectiveness of the installation of office automation technology. 


C. USEFULNESS OF EVALUATIONS 
In general, evaluations are decision tools; they are an aid in 

making choices (House, 1980). With regard to the issue of 
incorporating office autamation technology at Naval Laboratories, a 
government manager, executive, or official could use an evaluation as 
a aid in making choices about providing office autamation equipment to 
Naval Laboratories. However, evaluations have a special usefulness to 
government employees as written by Franklin and Thrasher (1976): 

In an era of heightened public concern about the distribution and 


management of public funds, at a time of diminished buying power, 
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when 1ldeology of a “good return on an investment" competes with more 
charitable and affluent ideologies for ascendancy in the public 
ethos, evaluation becanes a prominent and visible concern for the 
managers of public programs. (p. 2) 

In a democracy ultimate accountability is to the public; the public, 
by direct expression and through its elected representatives, is 
increasingly demanding a rendering of that account. (p. 3) 

According to Joseph Wholey (1979), those in charge of government 
prograins must make a countless number of decisions under tremendous 
time constraints, often with little reliable information on program 
performance. He writes that the key to better decisions and better 
government programs is the establishment of realistic and measurable 
objectives and measures of program performance and the use of program 
performance information to bring about changes in program activities 
that will ultimately enhance program performance. In short, Wholey 
advocates using program evaluation as the major decision aid for 
Severment Managers. 

In summary, an evaluation of office automation systems at Naval 
Laboratories is the measurement of desirable and undesirable 
consequences due to OA installation. The purpose of an evaluation of 
Naval Laboratory OA systems is to obtain evidence about their 
effectiveness. This evidence is useful to government managers and 
executives who are responsible for Naval Laboratory Office Automation 


Programs because evaluations enhance program performance in an era 


when maximum efficiency and effectiveness is demanded by the public. 


D. TYPES OF EVALUATION 
Michael Quinn Patton (1978) writes that there are two main types 


of evaluation: quantitative and qualitative. He explains that these 
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two types of evaluation are due to the two basic paradigms that 
dominate evaluative techniques. The first paradigm, which he says is 
the dominant paradigm in evaluation, espouses the hypothetico- 
deductive methodology. This is more commonly known as the scientific 
method; it assumes that quantitative measurement, experimental design, 
and statistical analysis is the epitame of good science (Patton, 
1978). The second paradigm, which Patton calls the alternative 
paradigm, espouses the holistic~inductive methodology. This paradigm 
relies on qualitative data, holistic analysis, and detailed 
description derived from close contact with the targets of study 
(Patton, 1979). 

In what situation 1s 1t appropriate to use each of these methods? 
Patton (1979) writes that the hypothetico-deductive (quantitative) 
approach aims at performance and prediction while the holistic- 
inductive (qualitative) approach aims at understanding. Kuhn wrote: 
"the most deeply held values concern predictions: they should be 
accurate; quantitative predictions are preferable to qualitative ones" 
(Patton, 1978, p. 184). Patton (1978) contends that for predictions: 
"qualitative analyses in general have little legitimacy beyond certain 
limited exploratory situations" (p. 211). However, Patton (1978) does 
concede that there are situations where quantitative measurement is 
impossible (as in cases where an attempt to measure "goodness" is 
made) and where the qualitative method is the only practical method. 
He also states that to attain the goals set for some evaluations it 
may be necessary to use the qualitative method. 

Analogous to the two types of evaluation, quantitative and 


qualitative, is evaluation objectivity and subjectivity. Evaluation 
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objectivity is the ability to evaluate without the influence of 
personal feelings (House, 1980). Evaluation subjectivity is evaluation 
by personal feelings. Objective measures are distinguished froin 
subjective measures by the presence or absence of ostensive indicators 
(Guttentag and Struening, 1972). Ostensive indicators are hard data: 
data that 1s external to an evaluator/observer, data that can be 
specified and measured using some agreed upon measurement tools. 
Patton (1978) writes: 
Qualitative methodology and a responsive, illuminative approach to 
evaluation most frequently stimlate charges of subjectivity - a 
label regarded as the very antithesis of scientific inquiry. 
Objectivity 1S considered the sine qua non of the scientific method. 
To be subjective means to be biased, unreliable, and nonrational. 
Subjective data implies opinion rather than fact, intuition rather 
than logic, impression rather than confirmation. Scientists are to 
eschew subjectivity and make their work "objective and value free". 
(e726) 
This quotation from Patton should not be construed that opinion, 
intuition, and impression are mutually exclusive of fact and logic. 
The key point is that opinion, intuition, and impression are not 
confirmable. Evaluations should seek evidence that is ostensive and 
confirmable. 

Which of these two methodologies should be cnosen to evaluate a 
program? Patton (1978) wrote that different kinds of problems require 
different types of research methodology. NALTOACS "was establisned to 
coordinate the introduction of office systems technology into the Naval 
Laporatory as a means to enhance productivity." (NALTOACS, 1983, p. 1) 
The objectives for the NALTOACS Pilot Assessment, were to gather 


information relating to OA system planning, implementation, and 


benefits analysis (NALTOACS, 1983). The first two objectives, 
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gathering planning and implementation information, are questions of 
understanding; these objectives can best be attained through use of 
the qualitative methodology (Patton, 1979). The last objective, the 
benefits analysis, is a question of predicting perfomnance. This is 
the most important objective of the Pilot Assessment because it 
determines whether the project is worthwhile: no matter how well 
planned or implemented a system is the bottom line is cost/benefit; if 
the benefits do not exceed the costs then the system is not worth 
planning or implementing. To perform a cost/benefit analysis the 
quantitative methodology should be employed; it is the method that 
results in a true bottom line of dollars and cents and it is the 
method that provides confirmable results. 

The Pilot Assessment of the Naval Laboratories that had pilot 
office automation systems in place was an evaluation using the 
qualitative methodology. Appendix A contains a sample of the 
questionnaires and interview sheets used by NALTOACS to perform their 
evaluation. These questionnaires clearly sought the opinions and the 
perceptions of the system’s users and managers. This thesis does not 
take issue with the selection of this methodology to gather 
information about OA system planning and implementation. However, this 
methodology is inadequate to perform a cost/benefit analysis because 
it does not return ostensive data. Also, there are biases inherent in 
the tools of the qualitative methodology, questionnaires and 
interviews, that cause the results evaluations using this methodology 


to be suspect. These biases are discussed in the next section. 
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E. QUALITATIVE METHODOLOGY SOURCES OF INVALIDITY IN THE NALTOACS 
PILOT ASSESSMENT 


To this point information has been presented about the identity, 
purpose, and usefulness of evaluations in general. Also, the two basic 
types of evaluations have been presented with their perspective 
strengths and weaknesses with an argument why the quantitative 
methodology is preferable to the qualitative methodology for the 
performance of a benefits analysis. It has been established that the 
Pilot Assessment conducted by NALTOACS used the qualitative 
methodology: the questionnaires and personal interviews are subjective 
in nature (see Appendix A); tnis method invited opinion, intuition, 
and impression which is difficult 1f not impossible to confirm as 
fact. This section addresses the biases and sources of invalidity that 
result from surveys and questionnaires themselves. The NALTOACS Pilot 
Assessment 1s sited to exemplify the biases that explained in this 
section. 

Webb et al. (1966) list three broad sources of bias and 
invalidity when the qualitative methodology is used: 

1. errors that may be traced to those being studied, 

2. errors that come from the investigator, 

3. errors associated with sampling imperfections. 
It is beyond the scope of this thesis to address the latter two 
sources of evaluation invalidity. However, errors that may be traced 
to those being studied, called the reactive measurement effect by Webb 
et al. (1966), are discussed in detail. 

The most understated risk to valid interpretation is the error pro- 

duced by the respondent. Even when he is well intentioned and coop- 


erative, the research subject ‘s knowledge tnat he is participating 
in a scholarly search may confound the investigator’s data. Four 
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classes of this type of error are: awareness of being tested, role 
selection, measurement as a change agent, and response sets. (p. 13) 


Of the four of reactive measurement effect errors listed above, only 
awareness Of being tested, role selection, and response sets are 


applicable to this thesis; they are discussed below. 


1. The Awareness of Being Tested 
The awareness of being tested has been called the "reactive 


effect of measurement" bias by Campbell (1957). Selltiz et al. (1959) 


wrote: 


If people feel that they are "guinea pigs" being experimented with, 
or if they are being "tested" and must make a good impression, or if 
the method of data collection suggests responses or stimulates an 
interest the subject did not previously feel, the measuring process 
may distort the experimental results. (p. 97) 


This does not mean that all subjects that are aware that they are 
part of a test will give inaccurate responses. However, the 
probability of bias is high in any study in which a respondent is 
aware that he is the subject of a test (Webb et al., 1976). 

The subjects in the NALTOACS Pilot Assessment were definitely 
aware that they were part of an experiment. The "NALTOACS PILOT 
ASSESSMENT USER INTERVIEW CHECKLIST PACKAGE" (see Appendix A) contains 
the following instructions: 

The data being collected by this survey is 
required to conduct the "Pilot Assessment" 
directed by the Deputy Under Secretary of the 
Navy (Financial Management) and the Cammander, 
Naval Data Automation Command. 

Please take this opportunity to candidly express 
your opinions and needs so that the overall 
planning for office autamation (OA) in the Naval 
Laboratories can directly benefit from them. Also, 


please feel free to add any inputs which you may 
have and which are not covered by the checklist. 
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These instructions could influence the Subject to give responses so 
that they make a "good impression" as described above by Selltiz. 
There are at least two reasons for this. First, the Survey has been 
directed by the Deputy Under Secretary of the Navy who is in authority 
far above the respondents in the chain of command. This, coupled with 
the questionnaire requirement for respondents to sign the survey may 
have resulted in an unwillingness by the respondents to give responses 
that would "create waves". The second reason is related to the first 
in that the collector of the survey is from Washington and is, in some 
sense, in authority over the installation where the respondents work 
and the OA equipment is installed. There might be a tendency to give 
positive responses to avoid "telling on oneself" as an organization 
and incurring scrutiny from above. 
2. Role Selection 

Role selection is another way that a respondents awareness of 
research may produce bias. This bias may not be a defensive reaction 
or dishonesty but rather a specialized selection of a "proper 
behavior" by the respondent (Webb et al., 1976). The contention by 
Webb et al. here is that a person who is singled out by an 
experimenter forces the subject into a role defining decision, a 
decision as to what type of person should the respondent be to answer 
the questions. 

The "NALTOACS PILOT ASSESSMENT USER CHECKLIST PACKAGE" (see 
Appendix A) gives the respondent an opportunity to select a role even 
before the respondent takes the survey. After the instructions a "user 


profile" section asks the respondent for the following information: 
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USER PROFILE Date: 
Name: Code: 
Telephone: Bldg: 


How would you classify your job responsibilities: 


Canmand_and Support Technical 

me ExXeccueirve ___ Manager 

___ Manager __ Scientist 

a, SUPEEVISOF ____ Systems Engineer 
_____ Professional ___ Programmer/Analyst 
Eel onicail ge OEheu 

ee Oener 


The implication of this 1s that the respondents could "play a role", 
as suggested by Webb et al., and answer questions not as they 
personally feel but as they feel an executive, a manager, or a 
professional should answer the questions. 
Sa ISscomsis Sous 

Response sets deal with the way that questions are asked. For 
example, Sletto’s experimentation led to his assertion that 
respondents will endorse a statement more frequently than they will 
disagree with its opposite (1937). Campbell wrote that the direction 
of wording can definitely influence the respondents in questionnaires 
and interviews (1965). Webb et al. demonstrated that respondents 
generally have a preference for strong statements rather than moderate 
or indecisive statements (1966). They also demonstrated that sequences 
of questions asked in a very similar format produced stereotyped 
answers, such as a tendency to select a righthand or lefthand 
response. Further, they proposed that decreasing attention produces 
Diases from the order of presentation. 

The NALTOACS PILOT ASSESSMENT USER INTERVIEW CHECKLIST PACKAGE 


(see Appendix A) has questions that could cause response set errors. 
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There are questions that ask the respondent to choose an answer from 
among a list of applicable answers. However, 74% of the first answers 
listed were responses that were positive for office automation in 
general (see Appendix A, p. 68). For example: 


10. Please check the appropriate responses: 


CALENDAR 
Essential Available Use frequently 
Nice to have Not available Use occationally 
Do not need Do not know Do not use 


1f available 


I have _/have not been trained on this capability and 
I would classify the capability as: 


Adequate to my needs Easy to use 
Must be improved, Difficult to use, 
how so how so 


"Essential", "Available", "Use frequently", "Adequate to my needs", 
and "Easy to use" all possess a positive connotation while "Do not 
need", "Do not know if available", "Do not use", "Must be improved", 
and "Difficult to use" all possess a negative connotation. There are 
five pages of questions similar to the one above covering the various 
capabilities of the pilot office autamation systems. 


11. How do you rate the overall responsiveness of the 
system in operation? 


Seerast yameer 2 Sec) mee Sew (oan see) 
pee eeeuace | 2aascec) _.. Too slow (10 sec +) 


17. In what way has the system assisted you in the 
preparation of products? 


____ Get the job done with less effort, how much so? _ % 
____ Better quality of end products, how so? 

De Rewer sryPOs 

7 Beerer appeal ance 

____ More timely 
____ Other (specify) 
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Seventy percent of the questionnaire consisted of questions 
that were also asked in a similar format. The questionnaire was twenty 
pages long; fourteen of these pages (see Appendix A, questions 10 and 
18) were similar in format. Due to the length and repetition of the 
question format it is possible that stereotyped answers were given as 
Webb et al. proposed above. The sheer length of this questionnaire 
brings into play the Webb et al. assertion that decreasing attention 


can produce biases. 


F. SUMMARY 

This chapter has examined the NALTOACS Pilot Assessment 
qualitative evaluation methodology. This methodology was determined to 
be appropriate, in principal, to fulfill the first two objectives 
(gathering information about OA system planning and implementation) of 
the NALTOACS Pilot Assessment. However, this methodology was 
considered inadequate to fulfill the most important objective, a 
cost/benefits analysis. There are two reasons why the qualitative 
methodology is inadequate for a cost/benefits analysis. First, results 
are based on opinions which are not ostensive and confirmable in 
nature. For example, the NALTOACS Pilot Assessment made quantitative 
estimates of productivity gains due to OA incorporation without 
performing any quantitative analysis; inferences were drawn fram 
questionnaires and interviews - inferences of fact based on 
respondents” opinions. Specifically, the Pilot Assessment (NALTOACS 
1983) estimates a productivity gain of 8 to 12% and a savings per year 
of 1,681 to 2,415 man-years across the seven Naval Laboratories (p. 


49). This thesis does not contend the figures that NALTOACS presents 
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as inaccurate per se. However, there 1S no ostensive data here nor a 
method of confirmation. Thus, the conclusions drawn are suspect. 

The second reason that the qualitative methodology can result in 
erroneous evaluation conclusions is that the tools of this methodology 
(questionnaires and interviews) can, in and of themselves, bias the 
respondents. Thus, even if the methodological approach was appropriate 
to perform a benefits analysis the results would still be suspect due 


to these errors and biases. 


Zo 


III. PROPOSED OA BENEFITS ANALYSIS METHOD 


Chapter II critically analyzes the NALTOACS Pilot Assessment 
method. It begins with a discussion of evaluations in general, 
explains the strengths and weaknesses of the two main types of 
evaluation methodologies, and discusses evaluation objectivity and 
subjectivity. Then the NALTOACS Pilot Assessment methodology is 
examined; its inadequacy to attain its most important objective (an OA 
benefits analysis) is specifically addressed. 

This chapter proposes a detailed method through which an OA 
benefits analysis can be performed. This is done by studying the ATIPS 
program located at Naval Weapons Center China Lake, California. 

As previously stated in Chapter II, the corner stone of the 
quantitative methodology is the scientific method. The scientific 
method dictates that quantitative measurement and camparative analysis 
is the essence of reliable and valid evaluation (Patton, 1978). 
Measurement is the key, without it there can be no comparative 
analysis. The first task in tailoring the quantitative evaluation 
method into a specific application must be to select a measurement 
method that is consistent with the objectives of the evaluation. The 
evaluation objective that is the focus of this chapter is an OA 
cost/benefit analysis. Therefore, a method to measure the costs and 
benefits of an OA system must be devised. 

Prior to the installation of any program or system there should be 


perceived need for that program/system. This will be the starting 
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point for the OA system performance measurement criteria. Wholey wrote 
tnat an evaluation of any program or system must document the extent 
to which management has defined the measures of resource inputs, 
program activities, outcames, and impacts (1979). 
Tnere are two parts to the question of measurabdility: (1) the 
indicator of achievement and (2) the means of verification. It isa 
Standard part of program evaluation methodology that both indicators 
and means of verification be developed as part of any evaluatim. 
(Schrage et aim, [977) 

The implication of these statements is that performance 
measurement can and should pve done using management criteria as a 
pasis for evaluation. This type of approach to program evaluation is 
known as "goal achievement" (Franklin and Thrasher, 1976). Tne 
advantasges to the goal achievement approacn is that it allows the 
implementor to be the control for the study and it insures that the 
evaluation is measurable and relevant to the implementor’s original 
problems (Franklin and Thrasher, 1976). This approach is particularly 
useful in the evaluation of innovative programs (Rossi and Freeman, 
1982). Office automation systems in Naval Laboratories, specifically 
the ATIPS system, are innovative. Therefore, this thesis uses the goal 
achievement aporoach in its measurement of the performance of the 
ATIPS program at NWC China Lake. 

Rossi and Freeman (1982) write that the evaluation of an 
innovative program should include: 


1. Identifying and describing the problem of concern, 


2. Identifying the program and delineating the objectives developed 
to solve the problen, 


3. Designing an impact model, 


4. Designing an evidence gathering tecnnidue. 


oy 


This chapter follows this basic outline. The outline contains terms 
and concepts that may not be intuitively obvious; these terms and the 


concepts in each step are explained as they are addressed. 


A. PROBLEM IDENTIFICATION AND DESCRIPTION 

The following paragraph is taken from the ATIPS Concept 
Development Paper (NWC, March 1982): 

The Naval Weapons Center (NWC), China Lake, faces heavily increasing 
administrative workloads along with reductions in tne manpower 
currently handling those workloads. NwWc personnel work in an almost 
exclusively manual administrative environment. What is needed is a 
more efficient methodology for handling the necessary administrative 
tasks of Center personnel. (p. 1) 

An Inspector Generals inspection of NWC in March 1980 reported 
that manpower ceilings nad resulted in an increase in personnel 
workload and a decrease in the quality of work (NWC, July 1980). 
Through the 1970s tne workforce at NWC had been reduced twenty percent 
due to billet restrictions while the total budget and workload had 
increased twenty percent (NWC, July 1980). 

The problem that faced management at NWC Cnina Lake was how to do 
more work with less people without negatively effecting the quality of 


the end product. 


B. PROGRAM AND OBJECTIVES 

In response to the problems listed above, four alternative courses 
of action were investigated; the alternative that pranised the hicgnest 
return on investment in the shortest period of time was the 
coordinated development of an Automated Technical Information 
Processing System (ATIPS) (NWC, August 1981). As discussed previously, 


ATIPS is an office automation system that uses a central camputer and 
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remote access terminals to provide a variety of functions to the 


users. Among these functions are (NALTOACS, 1983): 


word processing electronic mail 
document processing electronic calendar 
business graphics form processing 


The objectives of ATIPS were delineated in the Concept Development 


Paper (NwC, August 1981). These objectives are listed below: 


1. Reduce administrative workload for technical and administrative 


versonnel. 
2. Reduce tne 


3. Reduce the 
retrieving 


4. Reduce the 
5. Reduce the 


6. Make NWa 
Evaluation 


amount of space reguired to store information. 


costs for typing, copying, transmitting, filing, and 
information. 


number and size of meetings. 
labor cost for each productive unit of work. 


more cost-effective Research, Development, Test, and 
Cenecta, 


7. Provide access to other NWC computers through a single terminal. 


8. Provide a direct cammunication route among all personnel wno 


require it. 


9. Provide ready access to large quantities of textual information. 


Another objective of the system (Objective 10) that was not 


communicated in 


the Concept Development Paper but was communicated to 


the author by the functional manager involved the quality of the 


deliverables that the Center produced; ATIPS was to increase the 


cuality of the reports, manuals, briefings, and documents that the 


Center produced. 


Greenwood “s 


(1984) contention that productivity 1s a neasure of 


efficiency and effectiveness can be applied to the stated ATIPS 


objectives. Efficiency is the comparison of inputs to outputs; the 


fewer inputs per unit of output (or the greater output per unit of 
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input) identifies an increase in efficiency. The first seven 
objectives listed for ATIPS are questions of efficiency; each of these 
objectives seeks to maximize outputs while minimizing inputs. The last 
three objectives are not concerned directly with efficiency (although 
that may be a side effect), rather these objectives are questions of 
effectiveness, of how well each worker and the Center can achieve its 
overall goal. 

The basic objective of ATIPS can be reduced to: increasing 
personal and Center productivity througn increasing both efficiency 


and effectiveness. 


C. IMPACT MODEL 
An impact model, which is similar to a logical model (Wholey, 
1979), is an attempt to translate notions regarding the 
inodifications and controls of a condition into a hypotheses on 
whicn action can be based (Rossi and Freeman, 1982). Tnere are two 
hypotheses that should be addressed: 
1. tne causal hypothesis, 
2. the intervention hypothesis (Rossi and Freeman, 1982). 
Tne causal hypothesis is a hypothesis about the influence of one or 
more processes on the condition that the program seeks to modify. It 
is the relationship between the program or system inputs, activities, 
and outcames (Wholey, 1979). For NWC China Lake the causal hypothesis 
is: The availability of only manual administrative tools coupled with 
an increasing workload and a decreasing workforce is responsible for 
lower productivity at NWC China Lake. 
Using this causal hypothesis as a base, tne renainder of the 
impact model can pe develoved through the intervention hypothesis 
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(Rossi and Freeman, 1982). Tne intervention nypothesis 1s a statement 


that specifies the relationship between a program, what is going to ke 
done, and the causal hypothesis. In the case of NWC China Lake there 


are three variables (manual administrative tools, increasing workload, 
and decreasing work force) in the causal hypothesis that influence the 
outcane (productivity). The only means by which to change the outcome 
is by changing one of the three variables. However, it was not 
possiole for the Center to hire more workers or to refuse projects and 
lighten its worxlcad. Therefore, the first two variables are 
constants. The only variable in which change was feasible was the use 
of manual administrative tools. ATIPS was purchased to replace manual 
administrative tools with automated tools. The goal was to increase 
worker productivity. Therefore, the intervention hypothesis is: Office 
automation tecnnology increases overall productivity at NWC China 
a. 

The imoact or logical model provides a clear view of the 
relationships between the initial conditions and problems, the 


intervening program, and the outcomes. 


D. EVIDENCE GATHERING/MEASUREMENT MODEL 

The evidence gathering technique that is used in an evaluation is 
influenced by the basic evaluation methodology and the evaluative 
approach. The basic evaluation methodology that this proposed OA 
cost/benefits analysis metnod uses is the quantitative method 
described in chapter two. The evaluative approach that this proposed 
OA cost/benefits analysis method uses is the goal achievement approach 


described earlier in this chapter. The implication of using the 
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Cuantitative methodology and the goal achievement approach is that the 
evidence gathering technique should yield ostensive data for 
comparative analysis by utilizing the program’s objectives and the 
program management ‘s projected performance measurement criteria. 
Therefore, the project ATIPS pcerformance criteria and tne program’s 
objectives are examined prior to the selection of an evidence 
gathering technique. Once the evidence gathering technique is 
determined tne measures of program performance are refined from the 
projected program performance criteria and the prosram’s objectives. 
Following that refinement evidence gathering technique can be 
impiemented. 
1. Projected ATIPS Performance 

NWC contracted The Mattox Group, a consulting firm in 
Pasadena, California, to study a portion of the workers at the Center 
for the purpose of identifying potential opportunities to increase the 
efficiency and effectiveness of the Centers managerial, technical, 
and support versonnel (Mattox, February 1981). The study consisted of 
three parts. First, a "Shadowing" was conducted of each member of a 
pilot group for 8 to 16 hours. Second, 250 representatives from all 
departments were interviewed. Third, interview data was correlated 
witn the "shadow" data (see Appendix B for Mattox Group evaluation 
data). Tne Mattox Group study concluded tnat an office automation 
system at the Center could have a tremendous impact and potentially 
save managers, technical and support starr 20% of their time. 

The ATIPS project development croup used this 20% figure as a 
basis to project the potential versonnel salary/wage cost avoidance 


savings that could be expected throughout tne life of ATIPS (see 
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Appendix B for the figures and calculations). The analysis indicated 
that ATIPS would result in a $5.7 M personnel cost avoidance in its 
first year of operation. This figure would increase as ATIPS grew to 
its full size and the life cycle cost avoidance for personnel could 
reacn $151 M (NWC, August 1981). 

The second area of quantitative measurement criteria invoives 
travel. The Mattox Group estimated tnat a 15% reduction in travel 
requirements would be possible through the use of ATIPS communication 
capabilities (Mattox, Fepruary 1981). NWC personnei were averaging 
7000 off Center trips per year at an average cost of $1,200 per trip 
(NWC, Aucust 1981). This meant that ATIPS could save $420 K in its 
first year of operation and $8.82 M over its life-cycle (see Appendix 
B). In addition to the reduction in the cost of travel the personnel 
cost avoidance that resulted from the decreased travel time was 
calculated. This would result in a $70 K cost avoidance in the first 
vear of ATIPS operation and a $1.4 M cost avoidance over the life- 
cycle (NW, August 1981) (see Appendix B). 

The project group also estimated that 50% of the Center’‘s 
costs for maintenance of office ecuipment woulda be avoided due to less 
wear and decreased usage of copiers, typewriters, and other office 
equipment. This would result in a $200 K savings the first year of 
ATIPS operation and a $4.2 M savings over the life-cycle of the 
system (NWC, August 1981) (see Appendix B). 

2. ATIPS Program Objectives 

The Goal achievement method of performance measurement 

requires that tne evidence gathering technique for a program be based 


upon the management ‘s measurement criteria and the orogram s 
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objectives and goals (Franklin and Thrasher, 1976). As presented 
previously, the ATIPS program objectives are: 


1. Reduce aaministrative workload for technical and administrative 
personnel. 


2. Reduce the costs for typing, copying, transmitting, filing, and 
retrieving information. 


3. Reduce the labor cost for each productive unit of work. 

4. Reduce the number and size of meetings. 

5. Reduce the amount of space recuired to store information. 

6. Provide access to other NVWC camputers through a single terminal. 


7. Provide a direct cammunication route among all personnel who 
require it. 


8. Provide ready access to large quantities of textual information. 
9. Increase tne quality of tne deliverables produced by the Center. 


10. Make NW a more cost-effective Research, Development, Test, and 
Evaluation Center. 


3. Evaluation Tecnnicue Selection 

Tne projected ATIPS performance criteria and the ATIPS program 
objectives are similar in tnat they both suggest that the introduction 
of ATIPS will result in either reductions or increases of costs, 
workloads, cost-effectiveness, and quantities of textual information, 
for example. This suggests that measurements made before the 
introduction of ATIPS could be compared to measurements made after the 
introduction of ATIPS; this comparison would result in indication of 
penefits realized/not realized due to the introduction of ATIPS. This 
type of evidence gathering technique is known as a quasi-experimental 
time series design (Franklin ana Thrasher, 1976). Wholey (1979) 
describes the quasi-experimental time series design of evidence 


gatnering. 
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. - . the evaluator measures outcame variaples several times before, 

and several times after, participants are exposed to the program. 

Program effectiveness is estimated by comparing the post-program 

measurements with the pre-program measurements. The evaluator plots 

the time series data; then the evaluator and other knowledgeable 

people examine the data, consider possible alternative explanations 

ror the observed results, and attempt to draw conclusions about the 

-Pececiveness of tne program. (p..160) 
This evidence gatnering technique fits the with the projected ATIPS 
verformance criteria and the opjectives listed for tne ATIPS program. 
Tne cuasi-experimental time series design is tne evidence gathering 
tecnnique that this proposed evaluation method uses. 

2.) Refinement Of Program Pertormance Measures 

Goal acnievement is tne evaluative aporoach used for the OA 
penefits evaluation metnod proposed by this thesis. As stated earlier, 
the goal achievement approacn requires that the program objectives and 
the orojected program performance measures be used as the basis for 
the evaluation. Tnis means that the selection of specific performance 
measures snould involve tne program’s objectives and projected 
performance measures. 
The ATIPS program oojectives are listed on page 9 of tnis chapter. 


Tnese 10 objectives can be canbined into five groups of objectives: 


1. Reduce the cost of administration by reducing the workload and 
reducing the cost per unit of output. 


2. Allow access through computer terminalis to multipole computers, 
large text libraries, and otner personnel. 


3. Reduce the number of meetings. 
4. Reduce storage space needed for information. 
5. Increase the quality of the work produced. 
The ATIPS program’s projected performance criteria are 
explained on pages 7 and 8 of this chapter. These performance criteria 
list three areas of performance measurement: 


So 


1. personnel cost avoidance, 

2. reduction of meetings requiring travel away from NWC China Lake, 

3. reduction in the cost of maintenance on office equipment. 
How do the objectives relate to tne performance measurement criteria 
establisned by management prior to the introduction of ATIPS? There 
appears to be only one direct relationsnip, tnat is between the 
objective to reduce meetings and the projected performance criteria to 
reduce travel due to meetinsgs. However, tnis projected performance 
measurement does not include meetings that are neld at NWC Cnina Lake. 

The cost avolaance performance measurement is only loosely 
related to the odjectives of cost reduction of administration and 
reduction of the numper of meetings. Cost avoidance as a performance 
measure does not lend itself to true measurement after the ATIPS 
program is in place. This is in contrast to the other two projected 
performance criteria which do lend themselves to measurement. 
Therefore, the author views cost avoidance as more of a goal of the 
ATIPS system rather than a performance measure for the program’s 
Stated goals. 

The cuestion remaining is wnat types of measures of 
performance should be used to provide evidence about the penefits 
optained from the incorporation of the ATIPS system? This thesis can 
only offer suggestions as to the appropriate measures of performance 
pecause to select these measures requires the involvement of the 
management (Rossi and Freeman, 1982). Wholey (1979) wrote: 

Tne key to useful performance monitoring is agreement between the 
evaluator and the intended user on the events to be :nonitored, tne 


measures to be used in the monitoring, and the intended uses of the 
resulting information. (p. 137) 
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This information leads the author of this thesis to suggest 
aDprooriate performance measures for each of the objectives of the 
ATIPS program. Each of the objective groupings on cage 9 are addressed 
in order. 

To measure the effect of ATIPS on the volume of paperwork at 
NwC (Objective 1) paper itself should be measured. A camparison of 
paper usage should yield a reduction in usace after ATIPS was 
introduced. This measure should be weighted to account for the 
ditferences in project load that NWC exveriences during the 
measurement veriods. 

To measure the the effect of ATIPS on the cost of 
administrative work (Oojectives 2 and 3) a comparison of 
administrative output and cost should ve made. Administrative output 
is not synonymous witn total paper usage. Output is tne sum of ail tne 
manuals, reports, priefings, presentations, letters, and the like that 
NWC China Laxe produces during a period of time. Cost 1s the total 
eculpment maintenance and labor costs during the same period of time. 
Labor costs are particularly difficult to measure pecause of the 
concept of cost avoidance. It is unlixely that tnere will be any 
Savings in personnel labor costs ver se. However, the concept of cost 
avoidance implies that personnel will complete tasks in a more turely 
fashion. The implication of this statement is tnat personnel should be 
apie to perform more tasks in the same unit of time. Therefore, since 
NWC is not expected to reduce its work force, to detect a change in 
labor cost per unit of work will require measuring workload and staff 


size: the outout should go up while the iaoor force remains constant. 
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To measure the effect that ATIPS has on the number and the 
size of meetings (Objective 4) the number of off station trips made 
due to meetings can be measured and compared (as suggested by the 
projected performance measurement criteria). Also, historical records 
could be used to determine the number of on station meetings required 
before ATIPS; the number of meeting required since the installation of 
ATIPS could be measured and a camparison made. 

A measure of the amount of space required to store information 
(Objective 5) could involve researching supply records to obtain 
information concerning the purchase of storage units before and after 
the incorporation of ATIPS. 

Measurement of the access to other camputers through ATIPS 
terminals, measurement of communications capabilities, and the amount 
of textual information available through an ATIPS terminal (Objectives 
6-8) can all be measured directly without research. 

Measurement of the quality of the deliverables produced by the 
center (Objective 9) before and after ATIPS could be done by direct 
comparison. Copies of deliverables produced before ATIPS incorporation 
could be campared with deliverables that are similar in nature but 
produced after ATIPS incorporation. The judges of the quality of the 
deliveraples could be a composite team of professionals fram both 
outside and within NW China Lake. Stufflebeam et al. (1974) wrote 
that professional judgement is an effective evaluation tool when 
measurement criteria are difficult to define and quantify. 
Professional judgement is based on the evaluators experience and 


expertise. The criteria of the quality measurement should be agreed 
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upon by the professionals prior to the actual performance of the 


evaluatim. 

The final objective is an objective that encompasses all of 
the others and includes the purpose of this proposed evaluation 
method: making NWC more cost effective. Measurement of the other 
opjectives will provide evidence for this objective. This is where the 
issue of cost/benefit is resolved. The measures described above 
produce figures of relative savings. These relative savings should be 
totaled and compared to the cost of the system. If the savings total 
1S greater than the cost of the system then the system is worth 


purchasing; it is worth purchasing for all the Naval Laboratories. 


E. SUMMARY 

This chapter describes an method of evaluating the impact of 
office automation technology in Naval Laboratories. The Naval Weapons 
Center China Lake, California is the subject for a case study; by 
utilizing information from NW an evaluation method is presented using 
the qualitative methodology and the goal achievement evaluative 
approach. The goal of the evaluation method is to provide ostensive 


data for the performance of a cost/benefit analysis. 
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IV. SUMMARY AND RECOMMENDATIONS 


This thesis presents a method for determining the productivity 
effects and the cost/benefit of office automtion in Naval 
Laboratories. Formulation of this method included analysis of the 
NALTOACS Pilot Assessment and an analysis of the pilot office 
automation system (ATIPS) at NWC China Lake, California. 

Chapter I presents a definition of office automation and provides 
evidence that organizations can realize both tangible and intangible 
benefits through the incorporation of OA technology. The primary 
benefit claimed for OA is increased productivity. However, a survey of 
current literature indicated skepticism over the realization of 
benefits due to OA. The chapter concludes with an overview of the 
NALTOACS Pilot Assessment and the ATIPS system at NW China Lake. 

Chapter II analyzes the NALTOACS Pilot Assessment method in 
detail. A qualitative methcd was used to perform the NALTOACS Pilot 
Assessment. The chapter provides evidence showing that this 
methodology was inappropriate for two reasons. First, the results of 
the qualitative evaluation are based on ceinions which are not 
cstensive and confirmable in nature. Second, the qualitative 
methodology can produce erroneous results due to the biases inherent 
in this evaluative paradigm. 

Chagter III presents a method of evaluating the impact of office 
automation in Naval Lakoratories, a method based on a quantitative 
method. This evaluation method incorporates the goal-achieverent- 


evaluative approach and uses the ATIPS system at NWC China Lake as a 
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subject of a case study. The goal of the evaluation method is to 
prcvide ostensive data for the performance of a cost/benefit analysis. 
The purpose of using the goal-achievement-evaluative approach is to 
insure that the goals, objectives, and performance criteria developed 
during project initiation are used in the project evaluation. 

The ATIPS office autamation program was estimated to cost nearly 
912 million. Incorporation of OA systems at all eicht Naval 
Laboratories cculd cost more than $100 million. One performance 
criterion for the investment of public funds is that the trustees of 
those funds maximize the return on the investment; this is 
responsibility of the position and demanded by the public (Franklin 
and Thrasher, 1976). To ensure that goal of maximizing return on 
investment 1s achieved, the implementation of evaluation techniques is 
required. These evaluation techniques should produce measurable and 
confirmable results. 

The NALTOACS Pilot Assessment was a qualitative evaluation of 
office automation in Naval Laboratories. The evaluation results were a 
compilation of the opinions of the surveyed respondents; inferences 
about benefits and productivity gains due to OA were made based on 
those opinions. The actual productivity results were not measured nor 
confirmed nor did they appear to be measurable or confirmable. Hence, 
the NALTOACS Pilot Assessment does not provide information which is 
useful in determining the program’s return on investment. 

The evaluation method proposed by this thesis produces results 
that are both measurable and confirmable. Further, the data produced 


by the evaluation method insures that the objectives of the system are 
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considered in its evaluation and that the program’s cost effectiveness 
1s determined. 

NALTOACS was established to coordinate the introduction of OA 
systems aS means to enhance productivity in Naval Laboratories 
(NALTOACS, 1983). The recommendation of the author is to perform an 
evaluation of an operating Naval Laboratory OA system using the method 
described in this thesis. This will provide ostensive data which will 
indicate whether office automation does indeed enhance productivity 


and whether that enhancement is cost effective. 
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APPENDIX A 


NALTOACS PILOT ASSESSMENT USER INTERVIEW CHECKLIST 
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NALTOACS PILOT ASSESSMENT 


USER INTERVIEW CHECKLIST PACKAGE 


The data being collected by this survey is required to conduct 
the "Pilot Assessment" directed by the Deputy Under Secretary of 
the Navy (Financial Management) and the Commander, Naval Data 
Automation Command. 


Please take this opportunity to candidly express your 
Opinions and needs so that the overall planning for office 
automation (OA) in the Naval laboratories can directly benefit 
from them. Also, please feel free to add any inputs which you 
may have and which are not covered by the checklist. 


Thank you for your cooperation and assistance. 


Project Manager 


USER PROPILE Date: 
Names. Code: 
Telephone: Bldg: 


How would you classify your job responsibilities: 


Command and Support Technical 
_... ~Executive 
_.. Manager _... Manager 

Supervisor Scientist 
_..._.-~Professional _.._. ~Systems Engineer 
_.._. ~ Clerical _..__ ~Programmer/Analyst 
—_— OEWOs a 


Other 


SURVEYOR TELE # NWC 


Did you use other keyboard devices in your office before the 
pilot OA was implemented? 


No. Yes __ Still use__ 
Typewriter aS Typewriter aaa 
Computer Terminal ___ Computer Terminal ___ 
WP Workstation = WP Workstation et: 


Personal Computer Personal Computer 


Do you have a personal computer at home? 
No Yes 


Do you have your own pilot OA workstation? 


No__, how far away is the workstation you share? Ft. 


Yes__, with 
—— Dedicated Letter Quality Printer 


—._.__ Shared Letter Quality Printer How far away?____ ft. 
__... Dedicated Dot Matrix Printer 

—__. -Shared Dot Matrix Printer How far away?__ ft. 
—... Shared Plotter How far away? Ft. 





Local Storage 


Do you need your own dedicated OA workstation? 


No Yes 


Do you or would you use it as a terminal for programming? 


No Yes 


How long have you been an active user of the pilot OA system? 


—_._ Less than 1 month _... Over 6 months 
—_.. Less than 3 months _... Over 12 months 
_.. Less than 6 months _... Over 18 months 


Do you feel that you were adequately prepared/trained to use 
the OA system? 
No. Yes__ Did not attend training sessions__ 


Do you feel the system provides enough OA self-help features 
to enable you to use it with ease? 


No__ Yes __, for frequently used capabilities 
No__ Yes _, for infrequently used capabilities 


co 
AD 


Coe 


Do you feel the pilot project staff provides adequate user 
assistance? 


No Yas 


The following are general descriptions of the basic OA 
functional capabilities included in some pilot OA systems. 
After each is a series of questions pertaining to your 
knowledge, use and assessment of them. (Please check the 
appropriate response/responses.) 


CALENDAR - Allows a user to maintain an appointment calendar, 
both public and private. It provides the functions for 


adding, changing, deleting, searching and viewing the 
entries. 


___Essential __Available __Use frequently 
___Nice to have __Not available _..Use occasionally 
_._Do not need _._Do not know if __Do not use 
available 

I have... /have not____ been trained on this capability and 
I would classify the capability as: 
___Adequate to my needs _._-™FEasy to use 
__Must be improved, __Difficult to use, 

how s0 ___ how so 


TICKLER Notifies the user on a given day and time that a 
certain action or task, previously entered by the user, needs 
to be performed that day. 


___ Essential __Available Use frequently 
Nice to have __\Not available __Use occasionally 
Do not need _.._DO not know if Do not use 

available 

I have___ /have not___ been trained on this capability and 

I would classify the capability as: 

_._Adequate to my needs ___Easy to use 

_._™Must be improved, __Difficult to use, 
how s@ Oe how so 











TELEPHONE - Provides the user with a standard telephone 
message pad for taking telephone messages for other users on 
the system. The message will be sent to the recipient 
electronically. 


___Essential ___Available Use frequently 


__Nice to have __Not available __Use occasionally 
___Do not need __Do not know if Do not use 


available 


in 
co 


I have___ /have not___ been trained on this Capability and 
I would classify the capability as: 


——Adequate to my needs __.Easy to use 
—__Must be improved, SE Oitficult to use, 
how so how s0 





DIRECTORY - Provides a convenient way for looking up the 
telephone numbers of associates that are frequently called. 


__Essential __Available Use frequently 
___.Nice to have —__Not available __Use occasionally 
__Do not need —__Do not know if Do not use 
available 

I have__. /have not___ been trained on this capability and 
I would classify the capability as: 
__.Adequate to my needs —_.__Easy to use 
__.™Must be improved, __Difficult to use, 

how so _ how so 


BULLETIN BOARD - Announces important events or information of 
general interest to all or a specified group of users. 
Announcements may refer to a longer explanatory record that 
can be retrieved by interested parties. 


__Essential __Available __Use frequently 

_.__Nice to have —__Not available __Use occasionally 

__Do not need ___Do not know if __Do not use 
available 


I have___ /have not___ been trained on this capability and 
I would classify the capability as: 








__Adequate to my needs __Easy to use 
___Must be improved, ___ Difficult to use, 
how so how so 
ELECTRONIC MAIL - Replaces the inter-office memoranda and 


other informal correspondence. It allows the uSer to scan 
his mail box, to read mail, to reply to the mail, and to file 
and print mail. 


__Essential __Available __Use frequently 

___ Nice to have ___Not available ___Use occasionally 

___Do not need ___Do not know if ___Do not use 
available 


I have /have not been trained on this capability and 
I would classify the capability as: 


__Adequate to my needs .___Easy to use 
___Must be improved, —— Difficult to use, 
how so __ how so 











- Provides a comprehensive input and editing 
capability that allows users to create, edit and manipulate 
text guickly and easily. 





___Essential ___Available —_.Use frequently 
_._\Nice to have __Not available __Use occasionally 
—_._.DO not need ___Do not know if —_._Do not use 
available 

I have___. /have not___ been trained on this capability and 
I would classify the capability as: 
__Adequate to my needs __Easy to use 
—__£™\Must be improved, __.Difficult to use, 

how™So .__._ ae how so 
FORM PROCESSING - Allows the user to retrieve a standard 


form, insert the information required and then store, print 
or forward the document. The user may also create special 
forms for one-time or repetitive use. 


__Essential —__Available —__Use frequently 
—_._.Nice to have __Not available _.__Use occasionally 
__DoO not need _.__Do not know if __DoO not use 
available 

I have___. /have not___ been trained on this capability and 
I would classify the capability as: 
__.Adequate to my needs _.__Easy to use 
—__Must be improved, __Difficult to use, 

how so ___ how so 
DOCUMENT PROCESSING - Extension of the basic word processing 


system to permit the pulling together of text from various 
sources to create a large document in a consistent format. 


__Essential __Available __Use frequently 

___Nice to have __Not available __Use occasionally 

__.Do not need _.__Do not know if __Do not use 
available 
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SCHEDULER - Determines the earliest time that all specified 
attendees are available by checking their individual 
Calendar records. Once all attendees have confirmed the date 


and time, the meeting will be scheduled and an entry will 
appear in each user's calendar. 


Essential —__Available —__Use frequently 
Nice to have —__.Not available —__Use occasionally 
Do not need —__Do not know if ___DO not use 
available 
I have__. /have not___ been trained on this capability and 
I would classify the capability as: 
__Adequate to my needs —_.__Easy to use 
—___Must be improved, __Difficult to use, 
NOW SO of how so 





How do you rate the overall responsiveness of the system in 
operation? 


__Fast (under 2 sec) —___Slow (5-10 sec) 


Adequate (2-4 sec) —__.Too Slow (10 sec or above) 


In what OA capabilities has response time been the slowest 
(list slowest first)? 


—~ 
e 


3. 


N 


A ee ee a 


On the average, what percent of your day is spent using your 
OA workstation? 


0-5% (1/2 hour) 
6-12% (1 hour) 
13-25% (2 hours) 
26-37% (3 hours) 
38-50% (4 hours) 
Over 50% 


Which capabilities do you use the most? 


Bee 
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15. Which capabilities would you use more extensively if there 
were more users on the system? 


1. 





EXAMPLES: Electronic Mail, 
Scheduler, Bulletin Board, 
etc. 


16. What specific functions or products do you use the system to 


complete? 
ine EXAMPLES: Correspondence, 
Technical Reports, Speci- 
Die fications, Memoranda, Admin- 
istrative Reports (by name), 
ane Statement of Work, Procure- 
ment Requests, etc. 
4. 
ao: 
6. 
cer 
8. 
9. 
Lge 


17. In what way does the system assist you in the preparation of 
these products? 


___.Get the job done with less effort, how much so?__% 


_._Better quality of end product, how so? 


_._.Other (Specify) 


__Fewer typos 
_.__Better appearance 
__More timely 
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18. The following are general descriptions of additional OA 
functional capabilities that may be installed in the future 


On your OA system. Please check the appropriate 
response/responses. 


CORRESPONDENCE CONTROL - Enables the tracking of formal 
correspondence as it moves from one addressee (location) to 


the next. It requires the posting of receipt and forwarded 
entries as it moves from one point to another. 


—_... Essential; would use frequently 

—_... Nice to have; would use occasionally 

_.. Do not need 

If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 


el 
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- Provides a central telephone directory-like 
listing available to all users which includes the name, 
telephone number, organizational code, and office location of 
all on-base personnel. 


__. Essential; would use frequently 
Nice to have; would use occasionally 
__. ~ Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 








PERT - Allows a user to construct and maintain PERT networks, 
and do CPM analyses. 


__. Essential; would use frequently 

__.. Nice to have; would use occasionally 
_._. Do not need 

If this capability is made available, specify which office 
Products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 
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INTERACT - Allows users to send and receive messages to each 
other in a dialogue fashion through their workstations. 


__._ ~ Essential; would use frequently 
—__.. Nice to have; would use occasionally 
__.. ~-Do not need 


If this capability is made available, specify which office 


products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 





ROUTING - Allows an electronic document to be routed 
sequentially to other users according to an input specified 
or pre-established distribution list. 


__... _Ess@ntial; would use frequently | 
___ Nice to have; would use occasionally 
Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 





- Allows the workstation to be linked toa 
computer system such that the workstation becomes a terminal 
on that system for programming, data retrieval, etc. 


__.. Essential; would use frequently 


_._.. Nice to have; would use occasionally 
__. Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 


SIGNATURE - Allows a user to “sign® a document electronically 
by typing in an alphanumeric string known ongy to 
himself/herself and the system. 


_.. Essential; would use frequently 
__. ~-Nice to have; would use occasionally 
__. Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 
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AUDIBLE MAIL - Allows the user to listen to a computer 
readout of his electronic mail, when the user is away from 
his/her workstation but has access to a touchtone telephone. 


—.__ Essential; would use frequently 


—__._ Nice to have; would use occasionally 
—_._.. Do not need 


If this capability is made available, specify which office 


products and their associated tasks would be accomplished 
more efficiently. 





Products Tasks 
VOICE MAIL - An audio version of electronic mail. Messages 


are input to the system by voice and recordings are retrieved 
by the addressees. 


___ Essential; would use frequently 
___. ~Nice to have; would use occasionally 
__.. Deo not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 
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TELECONFERENCING - A form of computer session where all 
attendees are simultaneously connected and communicate with 
each other in an interactive mode. 


Essential; would use frequently 


Nice to have; would use occasionally 
_... Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 





STOPLIST - Points out all occurrences of words in the text 
which have been prohibited. 


_..{_. Essential; would use frequently 
_.. Nice to have; would use occasionally 
_.... Do _ not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 
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ARCHIVE ~ Allows documents to be filed with user-specified 
filing codes, keywords, and indexes, such as subject, 
Originator, file number, serial number and date. Users can 


retrieve a copy or restore the documents to active (and 
modifiable) status. 


—__.._ Essential; would use frequently 


__._._ Nice to have; would use occasionally 
Do not need 


If this capability is made available, specify which office 


products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 
FILE TRANSFER ~- Addresses and transfers stored text or data 


files from one computer storage system to another, or from 
one station (assuming local storage) to another, etc. 


Essential; would use frequently 
__.. Nice to have; would use occasionally 
Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 














- Allows the user to construct 
columns and rows of numbers. Column and row totals are 


automatically corrected as individual members in the matrix. 
are modified. 


—— Essential; would use frequently 
Nice to have; would use occasionally 
Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 

















DATA FILE - Allows the user to create, edit, maintain and 
retrieve data in a format (with edit criteria) as specified 
by the user. 


—_... Essential; would use frequently 
_._... Nice to have; would use occasionally 
_._._.. Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 


REPORT WRITER - Allows a user to subject a data file to 


elementary humeric manipulations and to retrieve the results 
in a report format specified by the user. 


—__._ «Essential; would use frequently 


__.. ~Nice to have; would use Ceres ty 
__.. ~ Do not need 


If this capability is made available, specify which office 


products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 


INTERR TE - Allows a user to display a record(s) meeting 


specified selection criteria from a data file specified by 
the user. 


_._.. ~ Essential; would use frequently 
__ Nice to have; would use occasionally 
Do not need 


If this capability is made available, specify which office 
products and their associated tasks would be accomplished 
more efficiently. 


Products Tasks 
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20. 


21. 


22. 


23% 


24. 


29. 


To what level would you increase your usage of the system if 
it offered al] the capabilities described above. 


0-5% (1/2 hour) 
6-12% (1 hour) 
13-25% (2 hours) 
26-37% (3 hours) 
38-50% (4 hours) 
Over 50% 


ATT 


How would you rate the reliability of the hardware? 


_._ ‘~ Excellent __.. Good —_.. ~- Satisfactory _... Poor 


What hardware devices have required the most maintenance? 


CRT 

Keyboard 

Printer 

Disc Drives 

Telecom Interface or Telephone Lines 
Other (Specify) 


If hardware failure occurs, how would you rate the response 
time for maintenance? 


__.. Adequate __. —Slow __. ~‘Too Slow 


Do you have a personal contact for questions and problems 
concerning OA software applications? 


No Yes 


In-house ___ 
Vendor 


How often have you experienced software problems? 


Rarely (Once every 3 months) Often (Twice a month) 
With what capability: 
__.. Occasionally (Once a month) 





How would you rate the response time for software 
maintenance? 


__. Adequate __.. Slow _... Too Slow 
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How often does a system failure or crash occur? 


Rarely —_— Occasionally Often 


(Once every 3 months) (Once a month) (Twice a month) 


Are there backup and recovery procedures in the event of 
system failures? 


No ___ Yes 


Do you have to repeat (or re-enter) work already done after a 
System failure or crash? 


No Yes 


Does the potential unreliability of the system inhibit you 
from using it for quick response or fast turn-around? 


No Yes 


(Please comment) 


a a eee 
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Are you given satisfactory notification of scheduled 
downtime? 


No Yes 


What are the most freguent reasons for the scheduled 
downtime? 


Scheduled maintenance 
Telecommunications 
Software enhancements 
Hardware modifications 
Other (specify) 


Do you or would you like to use your workstation for non-OA 
functions? 


Programming 
Data retrieval 
_.._.. Other (Specify) 








33. Do you by-pass the OA functions and use your workstation to 
directly interact with system software? 
No 


Yes (If yes, why) 











34. To what level would you increase your cumulative usage of the 


system if your workstation accumulatively added these 
features: 


USAGE 


06-12% 13-25% 26-37% 35-50% 51-62% 63-75% 
(1 he) (2 Kher) (3 Khe) (4 her) (5 Ker) (6 hr) 


ZEBEZBSBSBSBSSASEBESSBSSSSESESEBSSETSESBSBEESBESEESBSEBSBSESBTSESABZSESBESEBSBSEBSBESEBSBEBABBZABBEEBSEEsZs 
WORKSTATION 
WITH: 


Intelligence 
Local Storage 
Local Plotter 
Engin. Graphics 
Voice 

488 Interface 











35. What do you like the most about your OA system? 


36. What do you like the least about your OA system? 








Data Collection Monitor: 
Received 


Rev lewed 
Tabulated 


/s/ 
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This 1s a summry of the questions contained in the NALTOACS 
survey listed on pages 1 through 20. This summary examines each survey 
question to determine whether the responses in each question lists a 
response judged positive (by the author) for OA in general or 


negative/neutral for OA in general. 


Question 
Lou= oF negative/neutral 
IOs Series of 13 OA systems functions the respondent 
is asked to judge. Each of the functions has six 
response areas; five of these areas list positive 
responses for OA first. Therefore, there are 65 
positive and 13 negative/neutral 
He: positive 
12 - 16: negative/neutral 
i: positive 
iss Series of 17 additional OA functional 
capabilities that are available. Each function 
nas one positive response. Therefore, there are 
17 positive responses. 
LO: negative/neutral 
20: positive 
2 negative/neutral 
223 positive 
Za: negative/neutral 
2a 26: positive 
246 negative/neutral 
2em—20: positive 
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B0G=. 347 negative/neutral 
3a positive 
36; negative/neutral 
The NALTOACS questionnaire provides the opportunity for 124 
responses. Of that 124, 92 of the responses are judged by the author 
to be positive (in favor of OA in general) or the first response 
option listed is positive. The remaining 32 are considered negative or 
neutral of OA in general or the first response option listed is 
negative/neutral. Dividing the number of positive responses (92) by 


the total number of responses (124) yields a figure of 74%. 
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APPENDIX B 
PROJECTIONS OF POTENTIAL SAVINGS THROUGH THE 
INCORPORATION OF OFFICE AUTOMATION BQUIPMENT 
AT NAVAL WEAPONS CENTER CHINA LAKE, CALIFORNIA 
Table 1, Table 2, and Table 3 are graphic illustrations of 
estimates made by the Maddox Group to illustrate the opportunities to 
automate various office functions at NWC China Lake (Maddox, February 


aol) . 
Pages 73 through 77 of this Appendix are taken from the ATIPS Concept 


Development Paper (NWC, March 1982). 
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L; Personnel Costs. Direct costs are the salaries of personnel 
required to operate the ATIPS. The salary schedule for a technician is averaging 
$25,951 annually. For the purposes of this economic analysis, the figure of $25,951 
was inflated by 9 percent to account for cost of living increases between FY81 and 
FY82, adjusted by 10 percent to account for cost of living increases between FY82 
and FY83, then adjusted by 31 percent to conform to the guidelines set forth by the 
A-76 Cost Comparison standards. 


Average salary for FY81= $25.9 K 
Average salary for FY82 = $28.3 K 
Average salary for FY83 = $31.1 K 
Adjustment prescribed by A-76 31 percent 


Total adjusted average 
technician salary = $40.7 K 


The adjusted salary of $40.7 K is used as the salary of the ten 
personnel who are required for the operation of ATIPS. This staff will be used to 
operate and maintain the system, perform restart and recovery, and if time 
permits, training and helping users become familiar with ATIPS. 


Zs Maintenance Costs. Maintenance costs will not be incurred in 
FY83, since it is assumed that the first increment of the ATIPS will not be 
functional until FY84. Monies will be committed during the fiscal year requested, 
but delivery of hardware will be staged over time to facilitate installation and 
training. Annual maintenance costs are predicated on estimated maintenance 
contract costs of $250 per terminal, $5,000 per processing unit, and $10,000 per 
shared resource. The first increment of ATIPS (FY84) will require 60 terminals, 
three processors, and one OCR for a maintenance cost of $40,000. Costs in FY85 
reflect the use of an additional 480 terminals, 10 processors, one OCR, and one 
printer for an annual cost of $230,000. Costs for FY86 include the third increment 
of $40 terminals, 11 processors, one OCR, and one printer at an annual 
maintenance cost of $440,000. Costs for FY87 include the fourth and final 
increment of 540 terminals, 11 processors, one OCR, one quality printer, and one 
video dise at an annual maintenance cost of $660,000. 


0. Other Costs. Other costs include the annual costs for supplies 
(paper, ribbons, printwheels, ete.), equipment and building rental, if applicable, and 
for utilities (electric, heat, etc). 


4.2.5.2 ATIPS Benefits. Benefits are both quantitative in dollar savings and 
qualitative in terms of a more timely higher quality product. 


a. Quantitative Benefits. Quantitive benefits are derived from 
calculating dollar savings attributed to the more efficient operation provided by 
ATIPS, resultant cost avoidance savings and reduced expenditures of the current 
system. 


1. The results of a requirements analysis performed by The MATTOX 
Group in December 1980 through February 1981 at NWC, shows that ATIPS will 
result in a 20 percent savings in time. This savings will result in additional time 
being made available to a staff member to work on projects and is converted to 
cost savings by multiplying the 20 percent factor against the persons salary 
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(including all benefits), This value agrees with information from other similar 
office automation projects from both Government and industry and is felt to be 
conservative. Increases in efficiencies far above this 20 percent value have been 
reported in well organized and scheduled office automation activities. Industry 
publications indicate typical productivity increases of from 10 percent to 30 
percent using available systems. Amoco Production Research Co. claims a 60 
percent increase in the productivity of its personnel. Booz, Allen, and Hamilton 
estimates a 20 percent increase in productive time with the adoption of new office 
automation technology with the greatest payoff coming to senior managers in the 
form of enhanced decisionmaking. For the purpose of calculating cost avoidance 
based on the 20 percent savings, an average weighted salary (average of all salary 
classes including benefits in FY83 dollars) of $43.8 K is used. 


2. This value is obtained by proportionally averaging the number of 
members in each job category times their salaries. These salaries for each job 
category are: Scientist and Engineer ($35,241), Technicians ($25,950), Clerical 
($14,077), and Administrative ($26,634). The resultant average of $27,876 is then 
inflated by 9 percent to account for cost of living increases between FY81 and 
FY82, and by 10 percent for FY82-FY83 increases, and adjusted by 31 percent to 
conform to the guidelines set forth by the A-76 Cost Comparison Standards for 
benefits. 


Average salary for FY81 = $27.9 K 
Average salary for FY82= $30.4 K 
Average salary for FY83 = $33.4 K 
Adjustment prescribed by A-76 31 percent 


Total adjusted average 
salary = $43.8 K 


3. In FY85, it is estimated that 650 employees will potentially be 
affected by ATIPS from bringing on line 540 terminals and will realize an increase 
in efficiency of 20 percent at a cost avoidance savings of $5.7 M. This estimate of 
the relationship of employees to terminals is based on the known synergistic 
effects that come about through social interaction between employees (e.g., 
roommates sharing a terminal exchange information to the gain of each). In FY86 
an additional 950 personnel will potentially be affected by ATIPS from adding an 
additional $40 terminals. This results in an additional savings of $8.3 M or a 
cumulative annual cost avoidance savings of $14 M. In FY87 an additional 540 
terminals will be added providing use to an additional 900 personnel. This results in 
an additional savings of $7.9 M or a cumulative annual cost avoidance savings of 
$21.9 M. This annual savings will be the same for the remaining 5 years since no 
further terminals are anticipated after FY87 as a part of this purchase. Labor 
savings due to cost avoidance over the first 8 years of ATIPS in constant FY83 
dollars is $151 M. These savings are broken out for each year over the system's life 
cycle and are shown in Figure 4-7. 
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4. Other dollar savings can be achieved through a reduction in the 
travel budget made possible by the enhanced communications of ATIPS, reduction 
in the current maintenance costs of the status quo system, and reductions in costs 
for the Center distribution system (vehicle maintenance and operations costs). A 
breakout of these potential cost savings in FY83 dollars shown below are as 
follows: 


COST SAVINGS ELEMENT LIFE-CYCLE COST SAVINGS 


1. Off Center Travel $8,820 K 
2. Maintenance Costs of : 

Office Equipment $4,200 K 
3. Off-Center Travel Time $1,470 K 


3- The above savings in travel are attributed to an estimated 15 
percent reduction in traveling requirements made possible by the enhanced 
communications of ATIPS, allowing S&E's to communicate with off-Center 
personnel, data bases, and libraries, ete. NWC averages 7,000 trips per year in off- 
Center travel at an average cost per trip of $1,200. The study estimates that 
approximately 15 percent of this travel can be saved each year through the use of 
ATIPS. This amounts to $420 K the first year ATIPS is fully operational. Using 
FY83 dollars, the life-cycle (8 years) cost savings realized amount to $8,820 K. 


6. It is expected that savings will result from a reduction of time 
spent in a travel status. Assuming one day of lost time for each of the trips 
avoided above the savings the first year would be $70,000 and $1,470,000 for the 8 
years. 


7. Savings of the current maintenance results from decreased wear 
through less usage on such items as copiers, typewriters, and other office 
equipment. During FY 80, for example, NWC spent $260 K to maintain office 
equipment, $20 K per month on copy machine paper and $600 K on copy machine 
rentals. In the same period, $110 K worth of new word processing equipment was 
purchased and $139 K was spent to lease word processing equipment. These figures 
were converted to FY83 dollars and it is assumed that 50 percent of the related 
cost can be avoided. 


8. Additionally, it is expected that there will be a cost avoidance of 
$322 K (FY83 dollars) in FY84 in that the Center will defer anticipated purchases 
of office supplies and equipment to wait for ATIPS to supply the necessary 
capability. The savings due to cost avoidance over the expected 8-year life cycle 
of the ATIPS project total is $166 M, and is plotted by installation phase in 
Figure 4-8. This graph shows the acquisition costs and the total cost savings as the 
equipment is installed and becomes operational. A very early break even point (3 
months) is shown. However, a payback period of 1.25 years (the inflection point in 
the cost savings line) is claimed based on the sensitivity of the estimated 20 
percent productivity index rate. 
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